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ABSTRACT 

The major instrument employed in the evaluation of 
the functioning of the bilingual mini-school was an observation 
schema developed by the evaluators over a two year period. The major 
purpose of the observation scale was to develop a dynamic feedback 
system or process. Second, it was felt that it was important to 
obtain a direct measure of the processes going on in the classroom. 
In such a way it appeared possible to evaluate pupil learning, 
teacher and pupil behavior and interaction, methods and techniques 
employed, and the utilization of paraprofessional time. It was hoped 
that observations of the teachers associated with the school would 
occur bi-weekly, where the observers would be in the classroom for an 
entire instructional period, and that the observations would be 
representatively placed. In summarizing the results of the evaluation 
for the 1971-72 academic year, the program has proved successful in 
many aspects of its functioning, while at the same time not meeting 
many of the specific objectives of the program. The area which proved 
most successful was classroom process and progress, which is the most 
crucial aspect of the program. Pupils learned, were motivated, and 
demonstrated concrete academic progress. On the other hand the 
program itself may have been overambitious in stating some of its 
°?5ctives outside of the framework of the instructional component. 
(Autnor/JM) 
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BILINGUAL MINI-SCHOOL J.H.S. 45 



The role which the evaluators have assumed during the academic year 
of 1971-72 has been two-fold: first as observers of all aspects of the 
on-going program and second, as consultants to the project director and 
staff. The evaluation staff consisted of two principal investigators 
and a bilingual research assistant until March 1, 1972. After March 1st, 
the staff consisted of 1 principal investigator and the research assistant. 

The final report includes material presented in the interim report 
(March, 1972) and any additional information obtained from that date, 
in order to present a total picture of the academic year. Data will be 
presented within the context of each specified component of the program: 
Instructional, Materials, Guidance, Community Involvement, and Staff 
Development. Specific changes and suggestions for both the program and 
the evaluation procedure will be presented within the discussion of the 
specific components. 

In each of the components, data may prove to be incomplete. This 
data is +hat obtained from the individual classroom teachers. There has 
been continued difficulty in having the teachers furnish the evaluator 
with the necessary information required for the evaluation effort. The 
lack of complete information may require some modification in the evalua- 
tion design. Despite many steps which have been taken over this year and 
last, the present method of data collection may not be feasibie. It is 
believed that part of the problem arises from the whole concept of 
"evaluation." The staff does not recognize or accept the nece c ity for 
evaluation, and many regard it as an intrusion or interference in an 
innovative program, without seeing the need for evaluating any changes 
which have taken place. This is not a problem unique to the Bilingual 
Mini-School, but occurs in many programs where new methods and materials 
are being introduced. All too often, attempts are made to utilize 
materials and methods without actually evaluating what the input is. 
Newness is associated with change, when in fact, in many instances, more 
traditional methods and techniques might be equally effective.* To 
change attitudes toward the concept of evaluation would require a great 
deal of time, perhaps to the sacrifice of "innovation." Consequently, 
alternate procedures must be found in order to gather the necessary 
data required to meet the evaluation procedures. It is therefore 
suggested that the necessary data be collected as part of the program 
procedure rather than as part of the evaluation p. ocedure. For example, 



* An article which addresses itself to this question, but in a differ- 
ent content area is: Zach, L. and Kaufman, J. Is perceptual lag a 
meaningful concept. Journal of Learning Disabilities , June, 1972. 



each day attendance sheets are collected by the program director. In the 
same way, necessary material required in order to evaluate the program 
might be collected by the program director, either daily or weekly. The 
specific details of such a procedure need to be worked out with the pro- 
ject director in order to ease the burden of paper work. 



INSTRUCTIONAL COMPONENT 



1. Observation Scale 

The major instrjment employed in the evaluation of the functioning 
of the bilingual mini-school was an observation schema developed by the 
evaluators over a two year period (j copy is included in the Appendix). 
The major purpose of the observation scale was to develop a dynamic feed- 
back system or process. Secondly, it was *elt that it was important to 
obtain a direct measure of the processes qoing on in the classroom. In 
such a way it appeared possible to evaluate pupil learning, teacher and 
pupil behavior and interaction, methods and techniques employed and the 
utilization of paraprofessional time. It was hoped that observations of 
the teachers associated with the school would occur bi-weekly, where 
the observers would be in the classroom for an entire instructional 
period (45 minutes) and that the observations would take place during 
various times of the day, various days of the week and in a variety of 
subject areas. 

Observations were carried out by the principal investigator and by 
the bilingual research assistant independently. The observer would 
enter the classroom at *he beginning of the instructional period and 
remain seated at the rear of the room until the end of the period, with- 
out actually becoming involved in the classroom activity. The observer 
would take as ^lose to a verbatum account of what transpired in the 
classroom, with particular reference to those categories included on the 
bservation form. At the end of the observation session, the observer 
would fill in the observation sheet with the appropriate information. All 
observations were then summarized. 

For purposes of clarity and contrast, the results of observations 
cf the main teachers (those associated with the bilingual mini-school 
on a full-time basis) and auxiliary teachers (those who have loss than 
or part-time affiliation with the bilingual mini-school) were analyzed 
separately. An analysis of the results follow. 

A total of 184 observations were carried out since October 1971. 90 
of these observations took place in the classrooms of auxiliary teachers 
while 94 were in the classrooms of the main teachers. Figure 1 illustrates 
the distribution of observations for the months of October through June. 
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No observations took place during the month of September, since classes 
were being organized, children tested and placed in the proper class- 
rooms. It was noped that each main teacher would be observed twice per 
rr.onth and each auxiliary teacher, once per month. As illustrated in 
Figure 1, this goal was accomplished. During the month of June the num- 
ber of observations decreased because of the end of the year activity, 
i.e. graduation, final testing, yearbook work and record keeping. 

The results of the 184 observations were tabulated, calculated 
and are presented in terms of percentages in Table 1. Again the 
observations are separated in terms of auxiliary teachers and main 
teachers. An examination of Table 1 demonstrates that a great deal of 
variation exists among teachers regarding the methods, materials and 
techniques which are employed in the classroom. 

First, the physical set-up of classrooms was analysed. 78% of 
the auxiliary teachers employed a traditional set-up in the classroom, 
while only 28% of the main teachers employed a traditional set-up. 
Traditional refers to those situations where the children are seated 
in rows and the teachers' desk is at the front the classroom. The 
remainder of the teachers employed a non-traditional seating arrange- 
ment. However, even in those situations where a traditional method 
was employed, it was noted that the teacher would walk around the 
classroom, stopping at the student's desk and calling individual 
students up to his desk. Lessons observed covered all academic and 
non-academic subject areas. The greatest percentage of observations 
(45%) took place in science classes which include lectures, labor- 
atories and science club. Auxiliary teachers were involved in teaching 
music, typing, art, shop, English, Spanish as a second language, 
science, math and social studies. The teaching of Spanish to 
Spanish dominant students is an illustration of the great flexibility 
and awareness of students' needs within the program. This class was~ 
originally designated as an English as a second language class for 
the lowest academic level students. It was found that the children 
were not learning in such a situation and the students' academic 
Spanish was also quite poor. Therefore, the class was changed to a 
predominately Spanish one, where after working in Spanish (composition, 
discussion, grammar) the childrer then began to learn English. This 
flexible approach is quite characteristic of the program. 

Methods and materials employed in the classroom were examined. 
The most popular method of presentation of material for both groups 
of teachers was the blackboard. There appeared to be a less than 
expected use of textbooks, but a great deal of teachcrmade materials 
and experimental and concrete stimuli. In a great number of situations 
(89% and 92%) the methods and materials employed in the classroom 
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Table 1: Analysis of Observations in Percentages for Main and Auxiliary 
Teachers in the Bil ingual Mini -School (Auxiliary Teachers = 90; Ma Tri 
Teachers 109) 



Observation 



Physical Set-Up of the Classroom 



Results 


^Traditional 


Lecture 


Informal 


Semi-Circular 
Circular 


Auxiliary Teachers 


71% 


07% 


22% 




Main Teachers 


.28% 




18% 


54% 



* Traditional refers to classrooms where children are 
seated in rows and the teacher remains at the front of 
the room. Lecture refers to situations where more than 
one class is together. Informal refers to sicuations 
where students walk around, sit in informal groups, etc. 



Observation 



Content of the Lesson 



Results 



Auxiliary 
Main 



Spanish as 
a Second 
Math Language Science 



17% 
18% 



24% 



21% 
24% 



Spanish* 
Social to 
Studies ESL English Span. Pom 



16% 



21% 



22% 



Auxiliary 
Main 



Music Typing Art Shop 
2,5% 2.5% 2.5% 2.5% 



*This class was originally an ESL class, but students 
were doing poorly, so that it was changed to a 
Spanish class 
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Observation 


Materials and 


Media Employed (more than one 




category ma.^ 


t apjjlyi 








Auxiliary Teachers 


Main Teachers 


Results 








Blackboard 


86% 


87% 




Notebook 


16% 


22% 




Textbooks 


26% 


08% 




Experimental Materials 








and Concrete Objects 


32% 


48% 




Teacher-Made Materials 


52% 


85% 




Fi 1ms 


02% 


04% 




T.V. 




02% 




Reference Materials 




12% 




Pe nodical s 




21% 


Observation 


Are Materials Appropriate to Capaciti 


es 


and Interests of the Pupils? 






Auxiliary Teachers 


Main Teachers 


Results 










Seem appropriate: 


89% 


92% 




children motivated 








and attending to 








learning tasks 








Few interested 


03% 


04% 




Not interested 


08% 


02% 




Language too 




02% 




difficult 






Observation 


What Input Modes are Employed in Instruction? 








Auxiliary Teachers 


Main Teachers 


Results 


Visual 


100% 


100% 




Audi tory 


100% 


100% 




Oral Communication 


85% 


98% 




Writing 


33% 


52% 




Repetition 


23% 


37% 




Experimentation 


19% 


68% 




Questions 


42% 


84°/ 
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Observation 



Types of Responses Required of Students (cont'd.) 



Results (cont'd.) 



Auxiliary Teachers Main Teachers 



i. What are opportunities 

for self response 
j. Pupil experiments or 

projects 
k. Opportunities for creative 

problem solving 



Many Few 

92% 08% 
18% 

47% 



Many Few 



97% 03% 
42% 



89% 



Observation 



What is the Proportion of Time Devoted to 
Listening, Speaking, Reading and Writing 



Results 



Auxiliary Teachers Main Teachers 



Even distribution 


36% 


24% 


Predominately listening, 


38% 


42% 


responds to teacher's 






questions 






Copies from blackboard and/or 


06% 


06% 


text 






Discussion and oral communica- 


10% 


26% 


tion 






Mostly reading 


06% 


06% 


Drawing, painting and writing 


04% 





Observation 



Movies , Slides, T.V. 



Results 



Since in only 3 observations were movies or T.V. 
employed, these observations were not analyzed. 



Observations 



Does Teacher Keep Individual Records? 



er|c 



Results 



Yes 
No 



Auxiliary Teachers Main Teachers 

100% 
0% 



100% 
0% 
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Observation 



Is Homework Given? 
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Results 



Yes 
No 



Auxiliary Teachers Main Teachers 
£6% 76% 
MS 24% 



Observation 



What Type of Homework? 



Results 



Observation 



Auxiliary Teachers Main Teachers 



Carry over from lesson 
Problem solving 
Reading 

Writing composition 

Research 

Unrelated 



78% 
16% 
05% 
01% 



84% 
12% 



02% 



What Use Does Teacher Make of Homework? 



Results 



Observation 



Discusses and goes over 
in class 



Auxiliary Teachers M ain Teacher s 
100% 100% 



What Lanquage(s) Are Used in Classroom ? 







Auxiliary Teachers 


Main Teachers 


Results 


English only 


74% 


15% 




h English and ^Spanish 


04% 






English, some Spanish 


07% 


16% 




Spanish, some English 




12% 




Spanish predominately 


05% 


57% 
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Observation 



Under What Arrang em ent is the Class Taught? 



Results 



Auxiliary Teachers Main Teachers 



Individual groups 

Whole class 
(individual responses 
required) 

Individuals with 
individual super- 
vision 



24% 
72% 



04% 



42% 
50% 



08% 



Observation 



Results 



What Do Pupils Do While Waiting for the Teacher? 



Auxiliary Teacners 



Main Teachers 



Lesson Structured to 
include AWW and 
Does 

Children work from 
blackboard 

Involves as many as 
possible although 
some uninterested 

Disorganized. Teacher 
works while re- 
mainder talks 



81% 
02% 

11% 



86% 
06% 



Observation 



Types of Responses Required of Students 



Results 



a. Making inferences 
from pictures 

b. Making inferences 
from texts 

c. Finding factual answers 

d. Guessing at new words 

e. Reading aloud 

f . Imi bating teacher 

g. Imitating other pupils 

h. Giving associations to 
texts 



Auxiliary Teachers 



Main Teachers 



14% 


24% 


08% 


04% 


22% 


12% 


35% 


10% 


42% 


43% 


15% 


30% 


05% 


18% 


12% 


04% 
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Observation 



What Use is Made of the Paraprofessional? 



Results 



Observation 



None present 

Works with individuals 

Checks homework 

Discipline 

Translation 

Observation 

Record keeping 



Auxiliary Teachers Main Teachers 



86% 
04% 
02% 
02% 
02% 
02% 
02% 



28% 
48% 
02% 
05% 



16% 



Teacher's Understanding and Knowledge and 
Tolerance of Child's Cultural Background 



Results 



Understands needs 
Good teacher-student 
relation 

Does not seem to 
understand 



Auxiliary Teachers Main Teachers 



88% 
68% 

12% 



100% 
86% 



Observation 



Changes in Teacher Behavior 



Results 



Auxiliary Teachers Main Teachers 



Good relationship 78% 
Some discipline problems 06% 
Not attracting enough 06% 
students 

Teaching methods not 10% 

motivating 
Lack of organization 



88% 
04% 



08% 
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were evaluated to be appropriate to the interests and capacities 
of the students. The apparent lack which appeared in examining the 
types of materials employed was the use of audio-visual devices which 
have proved quite successful for motivation and learning in class- 
rooms. In only three observations were films or television employed. 
It would seem important to encourage the use of these techniques to 
provide variety and stimulation in classroom instruction. 

Results of observations indicate that the teachers stimulated 
predominant sensory channels in their structuring of materials. 
Again, research demonstrates that in situations where learners are 
having difficulty, the kinesthetic and tactile systems might prove 
effective aides to learning, and it is suggested that in a teacher- 
training session, a consultant might be employed to demonstrate ways 
in which these sense systems might be utilized to stimulate learning. 

The most popular technique for imparting information, is to 
teach to the whole group, requiring individual responses from stu- 
dents. Because classes are divided into levels (academic competencies) 
which are weekly evaluated, this might be the most effective method 
of instruction. However, opportunities for individualized instruc- 
tion should be explored. In the majority of cases the lesson is 
structured to include all students and in most cases does. 

A multitude of responses are required of the student in the 
classroom, stimulating many different levels of learning. It is 
interesting to note that a significantly greater percentage of time 
is provided for creative problem solving with teachers primarily 
associated with the mini school when compared to the auxiliary 
teachers 89% to 47%. Many opportunities are provided for self- 
response and intrinsic reinforcement. 

Although the stated goal of the program is to provide equal 
amounts of time in listening, speaking, reading and writing, it is 
found that this varies from classroom to classroom, where different 
abilities are stressed in relation to the material being taught. 
However, because of the great variety of teachers that students 
are exposed to, it would appear that none of these vital areas are 
ignored. 

Homework is given in the majority of classes, and in most of 
these the assignment is a direct carry-over from the classroom lesson. 
In all cases, children obtain reinforcement from the doing of home- 
work as it is gone over and checked in class the following day. 

The predominant language used by auxiliary teachers in the 
classroom is English, while with the main teachers it is to some 
degree Spanish. The degree of Spanish varies with the level of the 
students. 
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In 100% of the cases of the main teachers, there is an under- 
standing and tolerance of the child's cultural background and nt*ds, 
where with the auxiliary teachers there is a great majority of teachers 
aware of the child's cultural background. 

In most of the classes of the auxiliary teachers, there are no 
paraprofessionals present (86%). It would appear that in analyzing 
the way in which paraprofessionals are used in the classrooms, that 
they are not being used in the most effective way. Much paraprofession- 
al time is spent in record-keeping in the classroom. It was found in 
situations where paraprofessionals were used to help with individual 
students they were quite successful. It is recommended that the pro- 
ject director reassess the way in which teachers are usinq their nara- 
professional time. In addition, it might be helpful for 
to be present in the classes of auxiliary teachers. 

In summarizing the results of the observations, it can be seen 
that the main teachers in the program employ more innovative techniques 
in the classroom, use the Spanish language with more frequency, pro- 
vide more opportunity for creative responses and problem solving. 
However, much of the techniques used are quite traditional and still 
the children are learning, are motivated and attend classes, when con- 
trasted to students in mainstream junior high school classes. What 
qualities then make the difference in the bilingual mini-school? 
Major contributory factors appear to include the fact that teachers 
are cognizant and ~. the cultural background of the children 

they teach and incorporate this into the materials they use in the 
classroom, i.e. Spanish language newspapers, social studies lessons 
which deal with the neighborhood, problems of immigration, thus 
making the material more relevant to the students needs. In addition 
most teachers have some knowledge of Spanish and can communicate with 
children in the native language. 

There is much information obtained from observations which cannot 
be tabulated. A key point must be mentioned. While children who 
are Spanish speaking continue to learn both in Spanish and English 
many children who did not speak any English at all at the beginning 
of the program are beginning to employ their second language in 
informal conversation. Many children are demonstrating language 
flexibility and are able to switch easily from one language to 
another, thus becomi. j truly bilingual. 

Directly pertinent to this point, is information obtained from 
interviews with the teachers of Spanish as a second language. Both 
teachers reported that although academic achievement for English speak- 
ers within the mini-school and children outside the mini-school might 
be comparable, there is a marked difference in conversational abilities 
in the two groups. English dominant children in the bilingual 
mini-school can actually carry on conversations in Spanish after a 
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minimum amount of instruction. This is attributed to the fact that 
they are in close association (official class, group activities) with 
Spanish dominant children and attempt to communicate in Spanish. 

The bilingual mini-school is a school within the school. Children 
have developed a sense of loyalty to the program and a feeling of 
belonging, which contributes to their involvement in the learning 
process. 

It is felt that the observation form developed by the evaluators 
has proved to be an effective instrument. It has permitted a first 
order observation of the actual learning process and what actually 
transpires in the classroom. Thus the evaluators have actually felt 
part of the educational situation. 



B. Pupil Progress Records: 

It was the goal of the evaluators to maintain ongoing progress 
records of pupil achievement in the classroom based upon the results 
of teacher-made tests. In addition, the teacher-made tests were to 
be evaluated by independent observers (teachers who were teaching on 
the same grade level in the same subject areas) as to their meeting 
of the stated objectives of the tests. Information obtained is in- 
complete, because of insufficient information provided by the pro- 
gram. However, the results of the information which was obtained 
has been analyzed and is presented. 



TABLE 2: EVALUATION OF TEACHER-MADE TESTS 
IN TERMS OF STATED OBJECTIVES 



Subject Math 
Area 



Inter- 
judge 
reli- 
ability 

Percen- 
tile of 
agreement 
with 
stated 
objectives 



,98 



95% 



Social 
Studies 



,76 



75% 



English as 
a Second 
Language 

.86 



Science 



.88 



92% 



Spanish 



.91 



94% 



ERiC 



Bl. Tests in each subject area, along with the teachers 1 stated ob- 
jectives were submitted to two teachers in each subject area. The 
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teachers who volunteered for this task were from an urban area in 
Connecticut. They were asked to agree or disagree with the stated 
objectives of the tests after reviewing the material. Two measures 
were calculated. One measure indicated the amount of agreement be- 
tween the two judges in their evaluations. The area which produced 
the most disagreement was the area of social studies (see Table 2). 

The second measure taken was the percentage of agreement and 
disagreement between the judges and the teachers stated objective. 
Again the lowest area of agreement was social studies (Table 2). 
For the most part, the judges ratings indicated that the teachers 
were measuring on the teacher-made tests what they stated they were 
measuring. It can be seen that this was most prevalent in those 
areas where academic material is highly structured. In the area of 
social studies, the bilingual mini-school in many of their classes 
was attempting to develop an innovative social studies curriculum. 
Consequently, there may be some difficulty in the teacher's estimation 
of what he is teaching and what is being measured. It is suggested 
that the social studies curriculum be examined more carefully in 
terms of its specified objectives and what is actually being taught 
in the classroom. 

An analysis of teacher-made tests is an important one in terms 
of evaluating students' progress. It has been suggested in many 
research studies (Gage ed. Handbook of Research on Teaching ) , that 
results of teacher made tests correlate most highly with academic 
achievement. 



B2. In evaluating pupil progress records, one can conclude that 
students have achieved throughout the 1971-1972 academic year. 
Because of incomplete information obtained, it is difficult to 
present precise information. However, several inferences can be 
made about pupil progress. 

Based upon grades, children in the English dominant class 
are achieving on or above grade level when compared to other 
children. Achievement has been on a consistently high level (A and 
B range) and has been maintained throughout the school year. Apart 
from the Engl i h dominant students, there are pupils divided into 
four different levels of functioning. Level V pupils or the low- 
est performers have demonstrated the greatest increase in test 
grades (95%), suggesting that greatest academic progress took 
place in this group. For the other groups of students, children 
who began the school year performing in the 90-100% range continue 
to do so. What is of greater importance is that several of the 
children who began the school year performing in the 50-60% range 
are now performing in 90-100% range, where the tests are of equal 
difficulty. In addition, there has been much reorganization of 
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groups where many students have been upgraded to higher academic group 
levels. 

Based upon report card grades, the mini -school students represent 
a high proportion of the total school who have been placed upon the 
honor roll. All of the mini-school classes had honor students. In 
relation to the total number of students in the mini -school program 
when compared to the total number of students in the mainstream, the 
mini-school had more than twice the ratio of students on the honor 
roll. 



C. Standardized Testing: 

The greatest lack in the evaluation procedure is the paucity of 
information of childrens 1 performance on standardized tests. A 
Spanish language reading test was administered to all Spanish domin- 
ant children in November and again at the end of April (Prueba de 
Lectura). The tests originally scheduled by the program to be used 
were not employed. This particular test has no established norms and 
thus accurate grade level scores cannot be reported. 

In November, the following mean scores were obtained: 



7th grade Vocabulary Level of Comprehension 

Mean 14.8 (right) 13.5 (right) 

Standard Deviation 7.3 6.6 



8th grade 
Mean 

Standard Deviation 



Vocabulary 
17.2 
9.0 



Level of Comprehension 
11.5 
6.6 
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In April, the following mean scores were obtained: 



7th grade Vocabulary Level of Comprehension 

Mean 22.3 21.4 
S.D. 

S.D. 6.8 7.4 



8th grade Vocabulary Level of Comprehension 

Mean 26.5 22.8 

S - D - 9.5 10.2 



ERIC 



In examination of the mean scores from November to April there 
is a demonstration of progress as evaluated by this particular test. 
The program attempts to establish grade equivalents for the scores 
obtained and there is an indicated range of grade levels from 8.2 
to 3.4, with the mean grade level about 5.9. The evaluator and the 
project director were attempting to establish local norms for this 
particular test and at the same time searching for a comparable 
instrument which could provide some estimate of the Spanish domin- 
ant children's reading levels. There is a sore lack of "hard data" 
in the evaluation of the program. This may be attributed to the 
fact that there are few, if any, available tests to measure Spanish 
dominant childrens* performance, with local norms. There is a real 
need for such material, so that effective baseline data can be 
established. It is suggested that the bilingual mini-school should 
have an individual directly on staff who would be primarily responsi- 
ble for the development of evaluation instruments for the program. 

An additional factor which prevented the obtaining of "hard data" 
has been the difficulty of obtaining cooperation from the high school 
where many children have gone after leaving the program so that 
their progress might be followed. There are attempts to remedy 
this situation for next year. 

D. Additional data: 

Additional information has been obtained which would demonstrate 
the effectiveness of the bilingual mini-school program. Teachers 
O were asked to report students 1 visits to the library apart from 
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class pen ods. It was found that the students in the bilingual mini- 
school were among the most frequent users of the library. Their 
activities there included reading for pleasure and research. In a 
typical month 18 out of 23 students in an official class visited the 
library at least once. 

d o * Ea ? " eek >* ^ uestl ' on of the week related to science, mathematics, 
Puerto Ri can culture, etc. was posted on a bulletin board and all 
students were invited to respond to the questions. The most active 
participators were students from the bilingual mini-school. 

Students from the bilingual mini-school overwhelmingly partici- 
pated m the school 's science fair. 

The above factors demonstrate the degree of involvement of the 
children of the bilingual mini-school in academic learning. 

E. Suggestions for Implementation of Evaluation Procedure 

After being involved with the evaluation of the bilingual mini- 
school program it becomes increasingly apparent that techniques for 
the evaluation of bilingual education programs must go beyond the 
use of standardized instruments and techniques employed to date 
Observational data and test scores provide an indication of the 
direction of learning. However, as was pointed out previously, there 
is a paucity of standardized test materials specifically for the 
Puerto Rican child and in many instances the child is compared to 
chi.dren from completely dissimilar ethnic and economic groups Ob- 
servational indices provide useful information as demonstrated in 
this evaluation report. However this information reflects progress 
of learning, without actually focusing upon the process of fearm'nq 
perhaps unique to the bilingual child. The April 29, 1972 Saturday 
Review raises several important questions about the process es in- 
yolved in bilingual education. For example a study cited (p 58) 
indicated that "In Montreal, children who were educated bilinquallv 
scored higher on both verbal and nonverbal intelligence tests and 
appeared to have a more diversified set of mental abilities' than 
their monolingual peers." This point might be considered in light 
of the bilingual mini-school children. What effect does bilinqual 
education have upon mental operation, problem solving abilities, and 
flexibility in thinking? 

In order to completely evaluate the effectiveness of bilingual 
education programs in general and the program at J.H.S. 45 in partic- 
ular it is suggested that learning factors be investigated in a 
systematic manner by incorporating a research design into the evalua- 
tion design focusing particularly upon the conceptual abilities of 
monolingual (Spanish and English) children in relation to bi- 
lingual (Spanish/English) children. With the information obtained 
from such systematic investigation, it might be possible to obtain 
information which could be translated into curriculum and instruction- 
al techniques— the more information one has about the nature of 
the learner, the more effective education will be 
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MATERIALS COMPONENT 



One of the goals specified by the program for the 1971-72 
academic year was to set up a materials resource center. This goal 
was not accomplished. However, the project has attempted to gather 
all materials relevant to bilingual education. Through observation 
analysis, it was observed that the teachers in the program have 
independently developed materials which were relevant to needs of 
their particular students. Games, charts, plays, conversation 
materials were employed which captured and maintained the attention 
of students in the program. 

It is unfortunate that a teacher in one classroom was unaware 
of some of the materials developed by a teacher in another class- 
room. It is strongly recommended that the project director arrange 
sessions where teachers can communicate with one another regarding 
their personally developed materials and perhaps circulate them 
among the staff so that all students may have the advantage of 
experiencing these unique materials. 

As mentioned previously, there appears to be a need for 
integrating audio-visual materials into the program. 



Observations 
184 



TABLE 4 
Audio-visual materials used 
3 



percentage 

on 



As it can be seen from the above table, in only \% of the classroom 
situations was audio-visual materials used. It is recommended that 
the project director encourage the use of such materials. 



GUIDANCE COMPONENT 



Within the framework of the guidance component, it was the goal 
of the guidance counselor to work with classes of children and with 
individual children on a one-to-one basis. Several discussions were 
held with the guidance counselor in order to heln lier specify her 
objectives for the program. The evaluators have also worked with 
the guidance counselor in developing an interview form to be used 
with the parents of the children in the program. This form included 
gaining information regarding the child's developmental history, 
problems in adjustment, homelife, etc. At the conclusion of the 
program the guidance counselor had interviewed 75 parents, seeing 
some of these parents on an on-going basis. 

9 

ERLC 
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Group guidance situations were observed. Topics which were 
covered in these classroom situations included, desirable and un- 
desirable qualitites in people, the effects of drugs, how to get 
a summer job, what to do when being interviewed for a job. Dis- 
cussion ■> lecture and role play were employed as techniques in 
introducing the information to the students. The material was 
directly pertinent to students 1 needs and all students appeared to 
be excited by and motivated by the guidance sessions. 

Individual guidance sessions were held only in situations of 
crisis, rather than on an on-going basis. 

This is the first year of the guidance program. Prior to this 
year, the project director handled most of the guidance related 
situations and discipline problems. Unfortunately, because of the 
project director's former role, many of the discipline related pro- 
blems are still referred to her. The guidance counselor is be- 
coming more involved with these problems in order to remedy the 
situation. At the same time, the possibility of setting up an 
in-service workshop (to be conducted by the evaluator) focusing 
around issues involved with discipline is being explored. The 
guidance component is not as effective as it could be. There 
are many children who require contact on a one-to-one situation 
on an on-going basis. In addition, the guidance counselor re- 
quires an office whereby she can have more private communication 
with individuals who do come in and see her. Much of the parent 
interviewing could be handled by the paraprofessional staff so 
that the guidance counselor can spend more time with individual 
students. In addition, the guidance counselor should have greater 
communication with the paraprofessional s. 



PARENT INVOLVEMENT COMPONENT 

The evaluation team has been directly involved in assisting 
the guidance counselor and project director in the setting up of 
a weekly parent workshop. Forty parents were visited at their homes 
or invited to the school and interviewed by the research assistant 
and paraprofessional staff ard then invited to attend weekly work- 
shops. The evaluator, together ^vith the guidance counselor prepared 
the programs for the weekly sessions. These included visits from 
the Health Department, the Sanitation Department, varied school 
personnel, as well as informal "rap n sessions. The individual pri- 
marily responsible for conducting the workshops was the guidance 
counselor. 

Baby-sitting service was provided. Weekly reminders were sent 
out as well as having the paraprofessional staff visit the homes. 
An all-out effort was made to bring parents into the school. The 
attendance at the workshops unfortunately does not reflect the 
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amount of time and effort put in by the staff. The maximum attendance 
for any of the meetings to date has been seven parents. Weather 
conditions, household pressures, etc. may have interfered wi th 
attendance. On the other hand, parent workshops may not be the 
most effective way of bringing parents into the school. Records 
have been kept by classroom teachers of parents who spontaneously 
visit the classroom, or who are invited to school to discuss a child's 
particular problem. From the information handed in, about 60% of 
the children in the mini-school have had a family member in school 
to date. At the same time, the guidance counselor has seen at 
least one third of the parents of students in the program. This 
data seems to indicate that parents are willing to become in- 
volved in the school when it directly concerns their child. How- 
ever, one of the objectives of the program is to have parents 
become involved for purposes of more general education and to 
serve some of their own needs. Perhaps one way to have parents 
become involved is by beginning to hold special workshops at a 
community based location run by the school staf j: . Another option 
might be at someone's apartment or a recreation room of a local 
housing project. The content of the workshops may not be meeting 
the needs of the parents. Many have expressed the desire to learn 
English, and perhaps a workshop of this type might be more attractive. 
Prior to setting up any additional workshops, it is suggested that 
the parent needs be surveyed, through a questionnaire or interview, 
in terms of interests, hours available. 

Records were kept and submitted to the evaluators by the class- 
room teachers as to the number of parents who visited the classrooms 
and the purpose of their visits. When compared to the number of 
visits of parents at other Junior High Schools (obtained through 
talking to teachers teaching at this level) a significantly larger 
number of parents visited the bilingual mini-school classes. The 
reason for the visits included, discipline problems, asking advice 
about home and family, observing children in the classroom, dis- 
cussion of the students progress. The parents appeared to be 
willing to come to school, primarily because there were individuals 
who spoke Spanish in the school and because the staff understood 
the problems of the neighborhoods. 

When the forty parents were interviewed regarding their 
attendance at the parent workshop, some information was obtained 
about their knowledge and assessment of the bilingual mini-school. 
The majority of parents felt that the program was a "good thing" 
and that such a program has helped their children. Parents re- 
quested this year, as last year, a program in English as a Second 
Language for themselves. The bilingual mini-school has not net 
this commitment There appears to be motivation and interest 
on the parents' part which the program should capitalize on and 
make every effort to meet the needs of the parents. 
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The evaluators had hoped to do an intensive investigation of 
the relationship between parent involvement and pupil achievement 
in the program. Because of many factors, this could not be under- 
taken. The factors include; Minimum attendance at the Parent work- 
shops and insufficient information submitted by classroom teachers. 
A few observations, however can be made. Of the parents attending' 
the workshops, those who attended on a regular basis were parents 
who had high achieving students. Those parents who visited class- 
rooms on a regular basis, did so because of school and discipline 
problems their children were having. 

It is suggested that an extensive investigation of this nature 
be carried out in future years, in order to determine how uuch 
should be expended in bringing junior high school parents into 
school . 



STAFF DEVELOPMENT COMPONENT 



A good working relationship exists among staff members of the 
mini-school. Every effort has been made on the part of the staff 
to open channels of communication, particularly with regard to 
individual student progress. Weekly staff meetings are held. 
However, the minutes of these meetings have not been submitted 
to the evaluator. 

An all day conference workshop has been held. Many topics 
were included in the agenda of the conference, and many important 
areas were touched on. However, many details concerning day to 
day functioning of the program were brought up for discussion, which 
apparently had been left over from the school week, and consequently 
broader topics were overlooked. Unfortunately, because of daily 
pressures in teaching, it is possible that the teachers do not have 
the time to sit down and discuss many common problems. It is 
suggested that staff development conferences be scheduled somewhat 
more frequently, with a more structured agenda, so that many important 
areas can be discussed in depth. It might be possible to cover 
some of this material in in-service training courses. 

Teachers, both auxiliary and main, have been involved in 
taking courses at New York University. These courses include 
"History and Culture of Puerto Rico," "Oral Spanish for Teachers," 
"Curriculum Techniques and Audio Visual Aides," "Teaching English 
to the Foreigh Born," "Bilingual Education," "Theory and Practice." 
For the most part, those topics covered in courses have been 
incorporated into the classroom by the teachers, as noted in ob- 
servation. Teachers have developed an appreciation for the Puerto 
Rican culture and history and employ the Spanish language in the 
classroom. 
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There appears to be a need for a more developed and organized 
in-service training program. Through observation, several areas of 
training were noted, i.e. individualized instruction, audio-visual 
\ aides, discipline. If teachers can discuss these topics witnin the 

; context of the school, then they can put them into direct application. 



ADMINISTRATION OF THE PROGRAM 



Theoretically, the responsibility for the administration of 
the program is distributed among the staff with the ultimate responsi- 
bility for coordination resting with the director of the program. 
However, in actuality, it appears that the project director has been 
too directly involved in too many aspects of the program, without 
the distribution of the responsibilities* 

Since many of the goals and objectives of the bilingual mini- 
school have not been met, it can be hypothesized that the organiza- 
tion of the program may not be as efficient as it could, and that 
certain members of the staff (particularly the director) is over- 
burdened with tasks. It is therefore suggested, that prior to the 
start of the next school year, the division of responsibility be 
analyzed, and clearer lines of authority be set-up. It seems that 
the project director should be more involved with staff training, 
working with paraprofessionals , observation of classes, than with 
discipline problems and record keeping. 



SUMMARY 



In summarizing the results of the evaluation for the 1971-72 
academic year, the program has proved successful in many aspects of 
its functioning, while at the same time not meeting many of the 
specified objectives of the program. The area which proved most 
successful was classroom process and progress, which is the most 
crucial aspect of the program. Pupils learned, were motivated and 
demonstrated concrete academic progress. This is of primary importance. 
On the other hand the program itself may have been overambitious in 
stating some of its objectives outside of the framework of the 
instructional component. It may be unrealistic to assume that there 
will be a great deal of community involvement in a Junior High School 
program. Parents at this point of their child's education might 
be so disenchanted with school programs that they may not be willing 
to give a new program a chance. Objectives within the materials 
and guidance components have not been met. 
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Many real problems have arisen during the school year which may 
have interfered with the accomplishment of stated goals. Apart 
from the usual absenteeism on the part of the staff, one faculty 
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member was absent a great many times, due to a prolonged illness, and 
in fact, for the month of June was replaced by a new teacher. In 
addition, another staff member is leaving at the end of this yea^ to 
attend graduate school and the guidance counselor is also leaving, for 
a new position at a university. The project director was also out of 
the school building a great number of times during the school year, 
attending necessary conferences and meetings. However, despite these 
many interfering factors, it can be concluded that the bilingual 
mini -school accomplished its most important goal —students have been 
educated and can now enter the mainstream of education. 



APPENDIX A 

School 

Class 



Physical set up: 
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OBSERVATION SHEET 



Date 



Time of Day 



Observer 



ERIC 



Content of lesson: 



Materials and media: 



Are materials appropriate to 
capacities and interests of 
students? 



Input modes : 



Under what arrangement: (group, 
whole, individual- describe in 
detail ). 



What do pupils do while waiting 
to work with teacher? 



ENGLISH 



SPANISH 



T 
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OBSERVATION SHEET 








ENGLISH 


SPANISH 




Types of Responses Required of Students 




a. 


making inferences from pictures 








b. 


making inferences from text 








c. 


finding factual answers in text 








d. 


guessing at new word 








e. 


reading aloud 








f. 


imitating teacher 








g. 


imitating other pupils 








h. 


giving associations to text 








i . 


what are the opportunities for 
self-response? 








J. 


pupil experiments or projects 








k. 


opportunities for creative 
problem solving 








1. 


others 








Proportion of time devoted to listening, 
speaking, reading, writing: (Estimate 
pupil progress as well). 






t 

ERjC 

t— 
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OBSERVATION SHEET 

Movies, slides, T.V. ENGLISH 

a. where shown 

b. purpose (recreation, introduction 
of new subject matter, reinforcemenl 
of lesson already taught, etc.) 

c. classroom preparation before 
showing? 

d. classroom follow-up after 
showing? 

e. is level of material appropriate? 

f. is sound track comprehensible? 

g. any correlation with material 
read to class by teacher? 

h. new vocabulary taught? 
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OBSERVATION SHEET 



ERIC 



Teacher Behavior: 



ENGLISH 



Dobs teacher keep individual records 



Is homework given? 



Kind? 



What use does teacher make of it? 



What language(s) is the teacher 
using? 



Describe the role of the 
paraprofessional . 



Estimate teacher's understanding, 
knowledge and tolerance of child's 
cultural background. 



SPANISH 



Describe changes in teacher 
behavior (methods, materials, 
approach et. al.) since last 
observation* 
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APPENDIX B : FIGURE 1: NUMBER OF OBSERVATIONS OF TEACHERS 
ASSOCIATED WITH THE BILINGUAL MINI-SCHOOL 




_. 6 



2 



Oct. Nov. Dec. Jan. Feb. Mar. Apr. May June 

MONTHS 



Major Teach 
N=94 



Auxi 1 lary 
Teachers 
N=90 



